Fast Network Pruning and FeatureExtraction Using the Unit - OBS
نویسنده
چکیده
The algorithm described in this article is based on the OBS algorithm by Hassibi, Stork and Woll ((1] and 2]). The main disadvantage of OBS is its high complexity. OBS needs to calculate the inverse Hessian to delete only one weight (thus needing much time to prune a big net). A better algorithm should use this matrix to remove more than only one weight, because calculating the inverse Hessian takes the most time in the OBS algorithm. The algorithm, called Unit{OBS, described in this article is a method to overcome this disadvantage. This algorithm only needs to calculate the inverse Hessian once to remove one whole unit thus drastically reducing the time to prune big nets. A further advantage of Unit{OBS is that it can be used to do a feature extraction on the input data. This can be helpful on the understanding of unknown problems.
منابع مشابه
A Comparative Study of Neural Network Optimization Techniques
In the last years we developed ENZO, an evolutionary neural network optimizer which surpasses other algorithms with regard to performance and scalability. In this study we compare ENZO to standard techniques for topology optimization: Optimal Brain Surgeon (OBS), Magnitude based Pruning (MbP), and to an improved algorithm deduced from OBS (unit-OBS). Furthermore we compare results to a newly pr...
متن کاملFast Network Pruning and Feature Extraction by using the Unit-OBS Algorithm
The algorithm described in this article is based on the OBS algorithm by Hassibi, Stork and Wolff ([1] and [2]). The main disadvantage of OBS is its high complexity. OBS needs to calculate the inverse Hessian to delete only one weight (thus needing much time to prune a big net) . A better algorithm should use this matrix to remove more than only one weight , because calculating the inverse Hess...
متن کاملFast Voltage and Power Flow Contingency Ranking Using Enhanced Radial Basis Function Neural Network
Deregulation of power system in recent years has changed static security assessment to the major concerns for which fast and accurate evaluation methodology is needed. Contingencies related to voltage violations and power line overloading have been responsible for power system collapse. This paper presents an enhanced radial basis function neural network (RBFNN) approach for on-line ranking of ...
متن کاملSecond Order Derivatives for Network Pruning: Optimal Brain Surgeon
We investigate the use of information from all second order derivatives of the error function to perform network pruning (i.e., removing unimportant weights from a trained network) in order to improve generalization and increase the speed of further training. Our method, Optimal Brain Surgeon (OBS), is significantly better than magnitude-based methods, which can often remove the wrong weights. ...
متن کاملSkeletonization: a Technique for Trimming the Fat from a Network via Relevance Assessment, 6.3.2 Other Pruning Methods
Stock performance modeling using neural networks: A comparative study with regression models', Neural Networks. fully connected network, it is not sufficient to stop removing weights when b P() first increases. This is particularly true given that both SBP and OBD are greedy algorithms, and they neglect interactions between multiple inputs and weights. Thus, removing additional inputs or weight...
متن کامل